Web Survey Bibliography
In interpreting questions, respondents extract meaning from how the information in a questionnaire is shaped, spaced, and shaded. We carried out five experiments using five questions measured on a five point scale to investigate how the effects of visual heuristics affect the answers to survey questions and test whether effective question writing (using numerical and verbal labels) may overrule visual effects. The experiments were fielded in the CentERpanel. This is an Internet-based panel, although there is no need to have a personal computer with an Internet connection. If necessary, equipment is provided by CentERdata (the agency). The recruitment of new panel members is done through a random sample of landline numbers of candidates. The panel is designed to be representative of the Dutch population. We demonstrate that respondents use the “middle means typical” heuristic only in a polar point scale. When numbers 1 to 5 are added to the polar point format and with fully labeled scales, the effect of spacing between response options disappears. In a second experiment we show that respondents are confused when options are presented in an inconsistent order and do not follow the “left and top means first” heuristic. In a third experiment we find that inter-item correlations are higher in a polar point scale when items are presented on a single screen instead of separate screens. The effect gets smaller when numbers 1 to 5 are added to the polar point scale. In a fully labeled scale no effect is found. In a fourth experiment, we found no evidence that respondents use the “up means good” heuristic: the use of a decremental or incremental scale did not result in different answer distributions. In a fifth experiment, we show that respondents use the “like means close” heuristic: the adding of numbers that differ both in sign and value (-2 to 2) and the adding of different shadings of red and green color affects respondents’ answers. The effect of different numbers was apparent in both a polar point and fully labeled format, while the effect of color was only apparent in the polar point format.
Conference Homepage (abstract)
Web survey bibliography - Toepoel, V. (27)
- Using experts’ consensus (the Delphi method) to evaluate weighting techniques in web surveys not...; 2017; Toepoel, V.; Emerson, H.
- Data chunking for mobile web: effects on data quality; 2017; Lugtig, P. J.; Toepoel, V.
- Mobile-only web survey respondents; 2016; Lugtig, P. J.; Toepoel, V.; Amin, A.
- Online Surveys are Mixed-Device Surveys. Issues Associated with the Use of Different (Mobile) Devices...; 2016; Toepoel, V.; Lugtig, P. J.
- Doing Surveys Online ; 2016; Toepoel, V.
- The Effects of Adding a Mobile-Compatible Design to the American Life Panel; 2015; Toepoel, V.; Lugtig, P. J.; Amin, A.
- Higher Item Nonresponse Rates Caused by Slider Scales in Web Surveys; 2015; Toepoel, V.; Funke, F.
- Coding Surveys on their Item Characteristics: Reliability Diagnostics; 2015; Bais, F.; Schouten, B.; Toepoel, V.
- Investigating Response Quality in Mobile and Desktop Surveys: A Comparison of Radio Buttons, Visual...; 2014; Toepoel, V.; Funke, F.
- Informing panel members about study results; 2014; Scherpenzeel, A., Toepoel, V.
- Mixed-devices in a probability based panel survey. Effects on survey measurement error; 2014; Toepoel, V., Lugtig, P. J.
- Panel Conditioning in Difficult Attitudinal Questions; 2013; Binswanger, J., Schunk, D., Toepoel, V.
- Mobile devices a way to recruit hard-to-reach groups? Results from a pilot study comparing desk top...; 2013; Toepoel, V., Lugtig, P. J.
- Effects of Incentives in Surveys; 2012; Toepoel, V.
- Building Your Own Online Panel Via E-Mail and Other Digital Media; 2012; Toepoel, V.
- Recruiting A Probability Sample For An Online Panel: Effects Of Contact Mode, Incentives, And Information...; 2012; Scherpenzeel, A., Toepoel, V.
- How Visual Design Affects the Interpretability of Survey Questions; 2011; Toepoel, V., Dillman, D. A.
- Panel Recruitment via Facebook; 2011; Toepoel, V.
- Nonparametric Tests of Panel Conditioning and Attrition Bias in Panel Surveys; 2011; Das, M., Toepoel, V., van Soest, A.
- Response Quantity, Response Quality, and Costs of Building an Online Panel via Social Contacts.; 2011; Toepoel, V.
- Can Verbal Instructions Counteract Visual Context Effects in Web Surveys?; 2011; Toepoel, V., Couper, M. P.
- Words, Numbers and Visual Heuristics in Web Surveys: Is There a Hierarchy of Importance?; 2009; Toepoel, V., Dillman, D. A.
- Pictures in Web Surveys; 2009; Toepoel, V., Couper, M. P.
- Panel Conditioning in Web Surveys: A Comparison between Trained and Fresh Respondents; 2009; Toepoel, V., Das, M., van Soest, A.
- Visual Heuristics and Answer Formats in Rating Scales; 2009; Toepoel, V. Dillman, D. A.
- Design effects in web surveys: comparing trained and fresh respondents; 2008; Toepoel, V., Das, M., van Soest, A.
- Can I use a panel? Panel conditioning and attrition bias in panel surveys; 2007; Das, M., Toepoel, V., van Soest, A.